Allow users to set request option headers for Anthropic's LLMs #4785
Conversation
FallDownTheSystem
requested review from
RomneyDa
and removed request for
a team
✅ Deploy Preview for continuedev ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
There was a problem hiding this comment.
@FallDownTheSystem requestOptions is injected by this.fetch which uses the customFetch from the @continuedev/fetch package. This is pretty easy to miss
The anthropic beta changes look interesting, could you update it to just add them to this.requestOptions rather than spreading request option headers?
FallDownTheSystem
force-pushed
the
anthropic-headers
branch
from
1fee07e to
82b07e7
Compare
|
Oh I see. So the requestOption headers were always set, but they would overwrite the anthropic-beta header, so the prompt-caching wouldn't be set. I did get another idea, you could set the |
|
@FallDownTheSystem following up, apologies for the delays, these changes look like the right approach, could you check the conflicts? |
FallDownTheSystem
force-pushed
the
anthropic-headers
branch
from
82b07e7 to
9071259
Compare
Add special handling for anthropic-beta headers
FallDownTheSystem
force-pushed
the
anthropic-headers
branch
from
9071259 to
a3fa888
Compare
There was a problem hiding this comment.
All of the code itself looks perfect, thanks @FallDownTheSystem! Would you be able to quickly add a test for the setBetaHeaders method? It would be fine to do this in an adjacent Anthropic.test.ts file, make the method public, and just include ~2 test cases
|
Just wanted to check in on this. Definitely hoping to get tests in this PR, and if you need help just let me know! |
|
Hey @FallDownTheSystem, I'm going to close this PR for now since I haven't heard back. If you'd like some help getting the tests over the line, just let me know and I'm happy to help out |
Description
Allow users to set request option headers for Anthropic's LLMs and add special handling for anthropic-beta headers
Checklist
Screenshots
No visual changes
Testing instructions
Add
for the
claude-3-7-sonnet-20250219model, enable extended thinking, and ask it to think for as long as possible. You should see it go well past the typical 8k output token limit.